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Abstract 

This article extends results described in a recent article detailing a structural 
scale invariance property of the simulated annealing (SA) algorithm. These exten- 
sions are based on generalizations of the SA algorithm based on Tsallis statistics 
and a non-extensive form of entropy. These scale invariance properties show how 
arbitrary aggregations of energy levels retain certain mathematical characteristics. 
In applying the non-extensive forms of statistical mechanics to illuminate these 
scale invariance properties, an interesting energy transformation is revealed that 
has a number of potentially useful applications. This energy transformation func- 
tion also reveals a number of symmetry properties. Further extensions of this re- 
search indicate how this energy transformation function relates to power law distri- 
butions and potential application for overcoming the so-called "broken ergodicity" 
problem prevalent in many computer simulations of critical phenomena. 

1 Introduction 

Recent advances in statistical mechanics have helped to explain the behavior of large 
ensembles of interacting systems. Since the time of Boltzmann, there has been a great 
deal of effort in attempting to understand the behavior of such systems. In his day, 
a great deal of progress was made using a form of entropy that seemed to explain the 
behavior of the so-called "ideal gas" model, a model premised on the very limited form 
of interaction implied by elasticity assumptions. Notwithstanding these assumptions, it 
was able to predict, quite accurately, many attributes of the behavior of gases in thermal 
equilibrium. 
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Despite its enormous success for these idealized systems, the Boltzmann version of 
statistical mechanics has had limited success in explaining the behavior of more com- 
plex systems where the components interact in ways that are distinctly inelastic — that 
is to say, the particles do not obey simple Newtonian mechanics. In these models, 
the particles themselves may absorb energy during collisions in forms other than by 
changes in their velocity. The assumptions of the ideal gas model that these particles 
have no internal structure therefore cannot be assumed. Or, it may be that these par- 
ticles exhibit attractive or repulsive forces between them. In any event, their gross 
behavior is not well predicted in systems governed by the Boltzmann-Gibbs entropy 
formula: Sbg = -fc^ti 7Tjlri7rj where the iri represents the probability of the i' h 
energy partition and W represents the size (number of energy states) of the system. 
These complex systems obey various power laws and can exhibit phase transitions and 
related critical phenomena where there are drastic shifts in their energy configurations. 

These power laws and criticality properties seem to be quite diverse and ubiquitous 
in nature. To help explain these behaviors, Tsallis 1 10 11 1 developed a new entropy 
expression that forms the basis of a non-extensive form of thermodynamics: 

s q = — (1) 

where k is a constant and S q is the entropy parameterized by the entropic parameter q. 
In classical statistical mechanics, entropy falls into a class of variables that are referred 
to as extensive because they scale with the size of the system. Intensive variables, such 
as temperature, do not scale with the size of the system . Tsallis' form of entropy is 
non-extensive because the entropy of the union of two independent systems is not equal 
to the sum of the entropies of each system. That is, for independent systems A and B, 

S q (A + B) = S q (A) + S q (B) + a-g)^)^) (2) 

and Tsallis points out that q = 1 recovers the extensivity properties of the Boltzmann- 
Gibbs entropy formula and further, that lim g _>i S q = Sbg, hence can be seen as a 
generalization of Boltzmann-Gibbs 1101 1111 . Moreover, the stationary probabilities 
Pi (t) of the system being in some i further illustrates the generalization of the classic 
entropy form in that 

limpj(i) = 7Tj(i) (3) 

forallt > OfToln.4831. 

The entropic parameter q controls important aspects of the probability distribution 
of the energy levels making it is possible to markedly change the nature of the thermo- 
dynamic system being modeled. For example, when q > 1 the stationary probability 
distribution shifts from an exponential form to one with heavy tails that gives rise to a 

1 Combine two vessels of gas each with the same volume and pressure into another vessel of twice the 
volume, and the pressure and temperature of the combined gas will be the same as before. Energy and 
entropy, however, are examples of extensive variables in that combining several sources of either energy or 
entropy and you increase the total energy or entropy. 
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power law distribution, or conversely, one can increase the stationary probability of low 
energy states when q < 1, something quite useful in optimization (see e.g., ifTI 1 1 21 > . 

One of the best ways for studying the implications of this non-extensive form of sta- 
tistical mechanics is through the use of computer simulations. The simulated annealing 
(SA) algorithm provides these basic simulation tools as it is, at heart, a simulation of a 
thermodynamic system although it has been used principally for solving optimization 
problems. Tsallis developed a generalized simulated annealing (GSA) algorithm 1121 
based on maximizing the entropy in Q. 

Given these developments, it seems quite appropriate and useful to examine certain 
scale invariance properties 1 3 1 of the classic SA algorithm in light of the non-extensive 
form of entropy and the GSA. These scale invariance properties shed light on a number 
of behaviors and illustrate some curious effects on the configuration space itself. An en- 
ergy transformation function is identified and used to illustrate a number of symmetry 
properties and potential applications. In addition, the behavior of parallel forms of SA, 
also exhibit scale invariance properties and can be used to illustrate certain analogies 
in the behavior of large ensembles of interacting particles or systems, a basic aspect of 
non-extensive statistical mechanical systems. 

This article is organized as follows: Section|2]provides some background into non- 
extensive thermodynamics, simulated annealing, and SAs scale invariance properties. 
Section [5] develops the scale invariance properties of the non-extensive form of S A. 
These scale invariance properties indicate the existence of an energy transformation 
function. Section[5]describes certain asymptotic properties of this energy transforma- 
tion function that suggest how it turns an exponential distribution into a power-law 
distribution. Section [6] provides some discussion on the implications of the scale in- 
variance properties and the energy landscape transformation describes extensions of 
this work and directions for future research involving Markov Chain Monte Carlo sim- 
ulation of complex systems. 



2 Background 

2.1 Non-Extensive Thermodynamics and Generalized SA 

The GSA developed by Tsallis 1 12 1, entails a different form for the acceptance proba- 
bilities and stationary probabilities because of the form of Q, i.e., the stationary dis- 
tribution is based on maximizing the value of S q rather than Sbg- The latter leads to 
the well-known Boltzmann-Gibbs distribution 

e -fi/t 

Mt) = -j— (4) 

ABG 

where Zbg = Ylj=i is the normalizing Boltzmann Partition Function. Note 

that because we will often be referring to the SA algorithm and the Metropolis Algo- 
rithm, the value of the "energy" function will be denoted by /j which is typically used 
in optimization problems to denote an objective function value, but this can also denote 
some energy value. 
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Maximizing the non-extensive entropy subject to certain constraints, described be- 
low, gives rise to a stationary probability distribution 

where Z q = J2f=i [ l + (^T^) M ~ ■ Note that wil1 be used throu g nout to 
represent the stationary probability in the non-extensive case and 7T; (t) will represent 
the stationary probability in the classic, extensive case. 

The definition of the stationary probability pi (t) in l|5} is based on the particular set 
of constraints used in defining a thermodynamic system. Three distinct and noteworthy 
sets of constraints have been studied all of which employ the standard normalizing 
constraint X^ftC*) = 1- What distinguishes these constraint sets is the relationships 
they define between probabilities and energy or objective function values. Tsallis 1 13 1 
reports that different forms of these constraints have a number of implications for the 
non-extensive form of the probability pi(t). Here, the form defined by Tsallis' Type 2 
constraint 1 14 1 for GSA is used where, 

w 

Y^p q Ji = U {2) , a constant, (6) 

i=l 

whereas in his original paper, the constraint J^iPifi = U was used. This later con- 
straint was referred to as the Type 1 constraint in 1131 p. 537] and leads to a different 
stationary probability denoted here by pj. It bears emphasis that the limit in l|3} holds 
for all the different stationary probabilities that arise from the use of the different con- 
straints 1131 . Initially, the notions of scale invariant structures will be based on the 
Type 2 constraint and then later, in a more useful context, the scale invariance of the 
Type 1 constraint will be examined. 



2.2 Classical SA Scale Invariance 

Among the many interesting aspects of the SA and Metropolis algorithms is a scale 
invariance property associated with the stationary probabilities of various states. This 
scale invariance property is manifest in the identical mathematical forms of certain 
quantities involving individual states, aggregated states of the solution space, and the 
aggregation of states associated with multiple processors in an expanded state space. 
See (3]01 for a complete description. 

One aspect of this scale invariance involves the rate change of the stationary prob- 
ability of a state i with respect to temperature t: 

= ^[/i "</>(*)] O) 

where (f)(t) is the expected objective function value at temperature t. For aggregated 
states A = i^, ■ ■ ■ , i m }, define 

^(tjsjxt), /Aw ^y , (8) 
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the conditional expectation of the objective function value given that the current state 
is in set A (see 1 3 , p. 224] for a more complete treatment). It then follows that 



^ = ^ \Ut) - </>(*)] (9) 

where the similarity of Q and (|9} indicates a form of scale invariance. 
Scale invariance also extends to second moments. For standard SA, 

d(f)(t) df n (t) a 2 a (t) 



dt dt t 2 



(10) 



where cr^(t) represents the variance over the entire solution space at temperature t. 
Scale invariance is exhibited by the fact that 

dt t 2 ' 

and u\ (t) is the variance of objective function values at temperature t conditioned on 
the current state being in set A. See |3 p. 232-33] for details. 



3 SA Scale Invariance with Non-Extensive Entropy 
3.1 The Basis of Scale Invariance 

To demonstrate scale invariance based on aggregated states in the non-extensive case, 
a basis for making comparisons must be established. To that end, the following tem- 
perature derivative of Pi(t) is calculated for the non-extensive SA case. For notational 

i 

convenience, simplicity and to ensure positivity, let Ni(t) = [l + (^j^) fi] 1 ~ q (here- 
inafter we will drop the (t) from Ni (t) to further simplify the expressions) and taking 
the derivative of (|5}, 

a„ fj-\ 7 9 AT. _ il. , 



dPi(t) _ Z q &Ni-Ni&Z q 



Note that 



dN t _ d 
~~dT ~ dt 



fi 



fjN[ 
t 2 ' 



(12) 



(13) 



Z q = ■ Nj, and pi(t) — Ni/Z q and hence 



9 7 - d X-N -V^-VMl 
dt q ~ dt^ j ~ ^ dt ~ ^ t 2 ■ 

j J 3 



Substituting this and (TO} into ( Tf2l yields 
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r 2 



Pi(t) 
t 2 



9-1 



To further simplify and clarify, define 

to,?,*) s/i^vr 1 



fi N j 



(14) 



(15) 



and will often be denoted simply by ft or fi(t) where it is understood to involve fi, q 
and t. Substituting (1151 into dl4t and further simplifying yields 



dpijt) _ pi(t) 
t 2 



dt 



Mt)-(f)(t) 



(16) 



and which interestingly enough is analogous to |7}. Notice that for q = 1, fi(t) = 
fi, V t > 0. Tsallis 110111 1 1 established that lim 9 _,i pi(t) — iTi{t), hence, in the limit 
as q — > 1, ( 1161 — > 0. The symbol /<(t) will be referred to as the ^-transformation of 
/j. Later on, some interesting properties of this transformation will be examined. 



3.2 Scale Invariance from the Aggregation of States 

In similar fashion as in |3|, define pA(t) — J2ieAPi(t) f° r -4 C fl, where fl is the set 
of microstates and where W — and 



fA(t) 



PA(t) 



(17) 



the conditional expectation of fi(t) conditioned on the current state being in set A. 
Taking the derivative, 



dpAjt) _ d_sr (A-sr 
dt ~dt^ pm ~^. dt ■ 



(18) 



ieA 



ieA 



Substituting (Tf6j and (TfTJ into (TTJ} yields 



d PA (t) 
dt 



E 

ieA 



Pi(t) 
t 2 



fi(t)-(f)(t) 



^ PitMtt) v Pi (0 (/)(*) 

ieA ieA 

p A (t)^ Pl (t)Mt) PA (t)(f)(t) 
t 2 ^ 



ieA 



PA{t) 



t z 



PA(t) 

t 2 



[h® (/)(*)' 



(19) 
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where for aggregated states, dl9t has a similar mathematical structure as d!6i . hence 
exhibits a scale invariance property the foundation of which is based on the energy 
transformation function /»(/,, q, t). 



3.2.1 Scale Invariance in Second Moments 



Scale invariance in the non-extensive form of SA for second moments is indicated in 
the following where again the (/-transformation of /, and is used. First, the 

parallels to the classic case is illustrated. Thus, 



d(f)(t) 
dt 



dfnjt) 
dt 



d_ 

dt 



Pi(t)fi(t) 



dt 



E 

i60 



fi{t) +Pi(t)- 



dt 



Substituting into the first part of J20l > and simplifying yields 



dt 



t 2 



t 2 



dt ' 



Noting the form of the first two terms in (12 It and the fact that in the third term 



dfijt) 
dt 



g-i 
t 2 



(20) 



(21) 



(22) 



and substituting into (121 > and dropping the (t) for notational clarity and adding the 
symbol 17 to denote expectations over the entire state space yields 



d(f\ 
dt 



(/ 2 >n 



(f)n + (9 



l)(/ 2 ) f 



(q- 



t 2 



f 2 



(23) 



where cr^ represents the variance of the q-transformed objective function values over 
the entire objective function space (at temperature t). Obviously, this expression is 
slightly different from the classic SA case as indicated in HOI . Note that lim^i (Tq = 
er 2 } so that the expressions in d23i and J10I become equivalent. 

The expression in (I23> quite clearly shows the effects of the value of q — values 
greater (less) than 1 increase (decrease) the rate of change of the expected objective 
function values (^-transformed values). Eq. d23t also provides the basis for another 
form of scale invariance. Thus, after going through similar steps as in i20\ through 
(l23l we get 



dt 



12ieA Pifi 
t 2 pA 



a 
t 2 



1\ J2i£APifi 



t 2 



eA' 

PA 



(24) 
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Noting that the first and third terms indicate conditional expectations conditioned on 
the current state being in set A, then i24\ can be re-written in the convenient notation 

dh = {f 2 ) A -{f)\ + {<i-i){f 2 )A 
at t 2 

(<Z-1)(/ 2 >A 



A 



* ' (25) 
where (I25> is clearly analogous to (I23> and so exhibits a form of scale invariance. 
Again, in the limit as q — > 1 both of these equations correspond to the scale invariance 
of standard (Boltzmann-Gibbs) SA as indicated in i ll Ob and il Q , What is interesting 
however is the fact that the terms involving (q — 1) also scale with the aggregated set 
A in the non-extensive case. 

3.3 Scale Invariance from the Aggregation of Processors 
3.3.1 The Classic SA Case 

Fleischer GJ|6|P | describes another form of scale invariance based on the aggregation 
parallel and independent processes each running the SA algorithm independently of 
one another. This type of aggregation contrasts sharply with the aggregation of states 
and hence directly affects the nature of the scale invariance and how the algorithm 
itself functions. In this respect, one may view the SA algorithm as a single processor 
or particle that probabilistically visits a single state at each iteration of the algorithm. 
Thus, when a set of states is aggregated and fixed, the particle visits the aggregated state 
probabilistically and in a way that dictates how the stationary probability and objective 
function value should reasonably be defined, i.e., based on the sum of the stationary 
probabilities and the conditional expectation of the objective function respectively as 
given in (|8j. 

In aggregating p independent processors however, the state space becomes fl p 
and the stationary probability and objective function values associated with the set of 
processors must be defined differently and requires the following definitions. First, 
define a state in the product space spanned by p processors as iii 2 ...i p . In classic SA 
scale invariance 



(*) =!!*'<»(*) ( 26 ) 



m— 1 

and is based on the concept of the joint probability of independent processors. Simi- 
larly, the objective function associated with a set of p independent particles is simply 
the sum of each particle's objective function value as opposed to the conditional expec- 
tation, and defined by 

p 

fiii 2 ...ip — / , Jim ■ 



m—1 



Scale invariance is indicated by 

7T i 1 i 2 ...i p W = 



V •••V p-h\ii--ipl t 
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(see E]|6)) which has the same form as (0} and the fact that the temperature derivative 
is proportional to the difference between a function of the objective function value and 
its expectation: 

p| p.222] where the expectation is over all the states of the product space. 
3.3.2 The Non-Extensive SA Case 

Taking cues from this earlier work, define the stationary probability of state ii«2 • • • ip 
spanned by p independent processors and based on the Tsallis entropy by 

p 

Piit a ...i p (t) = Y[ Pi m (t)- 

m=l 

Simplifying using the case of p — 2 (the following expressions are readily extended 
to the more general case) and letting a = (q — l)/t for notational clarity, the joint 
probability of a state iii 2 is 



Piii2 (^) 



[1+gC/jl +fi2 +«/il/» 2 )] T ^ 

where f ili2 (t) = fa + f i2 + af^f^ and where it is also the case that 

2 



(28) 



w \ 

i=l / 
V W 

E E [* 



ii = l i2 = l 

Thus, the form of Pi x i 2 (t) in the product space closely follows that of Pi(t) with the 
advantage that the non-extensivity property is exemplified by j 2 (f ) which is some- 
what similar to the entropy relationships in 0. Note that for q = 1, j 2 (t) = fi x i 2 — 
hi +/i a f° r a H Thus, in the limit as g — * 1, J28t is equivalent to the Boltzmann-Gibbs 
case in d26t (recall 0). 

Examining the derivative of the stationary probability using (1 1 61 also reflects a scale 
invariance property and yields 

dPhh (t) d 
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Pijt) 



Pijf) 



t 2 



h - (f)t 



+Pi 2 (t) 



Ph(t) 



k - (f\ 



t 2 



t 2 

/iii2 — (f)n 2 



and in general it follows that 

dPii-i P (t) 
dt 



Pii-i P (t) r 
t 2 



fil- 



(29) 



where f^...^ = YL.=i fi m and (f)n» = P(f)n and equals the expectation of 
Note that d29l again has a factor equal to the difference between a function of the 
objective function values and its expectation and has the same structure as in 0, 0, 
CSJ, d and d27). 

Finally, let us examine the temperature derivative of (/}q p and see how it compares 
to J23l and Thus, 



0</>n, 9 



0t 



at 



Pii-ipitffii-ipit)- 



Using the results from ( I20t through (1221 and noting that 7^...* (t) is a sum of terms, 



a/«. 



a/ 



t 2 



t 2 



f 2 



yields 



a(/> 



(/ 2 )o P -(/)o P + (9-l)(/ 2 ) 



£2'' 



a/- 



/■ 2 

i 2 



i 2 

(g-l)(/ 2 )q P 
i 2 



(30) 



where <7q p represents the variance of the sums fi x ...i (t) (again, bear in mind that 
the dependence on t is not indicated). Eq. d30i has the same form as ( I20l > and d25l 
indicating a form of scale invariance. What is remarkable however is that in aggregating 
processors, the objective function is defined as the sum of the objective function values 
of each processor whereas in aggregating states, the objective function is defined as the 
conditional expectation of objective functions, yet they have the same form and in the 
limit as q — > 1 are equivalent to the aggregation of processors in classic SA. 

3.4 Consistency 

Fleischer 1 3 1 describes the concept of consistency in the scale invariant structure of 
SA based on the definition of the aggregated objective function /a- This consistency 
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property states that one can use identical means to define the objective function (here 
we describe it in terms of an energy level) associated with the aggregation of two or 
more aggregated states A and B. Thus, 



fAuB(t) = 



n A (t)f A (t)+Tr B (t)f B {t) 
ir A (t) +Tr B (t) 

The definition of the objective function for state A U B has the same form as the defini- 
tion of aggregated states A and B and thus the scaling phenomenon in these definitions 
is, indeed, invariant on all levels of scale. It is easy to see that this same consistency 
property holds for the transformed objective function (energy level) /aub based on 
values f A and fg in the non-extensive case. In non-extensive systems however, the 
concept of consistency extends a bit further than in the classical case because of the 
energy transformation function. 

The alert reader may have wondered, for example, if the definition of f A (t) f° r 
some aggregation of energy levels A defined in Mil could be equated to the ^-transformation 
of some value f A (t) obtained by the weighted average of the Mntransformed values /j. 
In other words, 

? ^icAPifc 1 fA nn 
J A = = T- f~- (31) 

Pa 1 + ajA 

where 

r ^2ieAPifi 

J A = ■ (32) 

PA 

In general, the weighted sum of transformed energy values is not equal to the trans- 
formation of the weighted sum of energy values. The exception is when q = 1 or the 
temperature is infinite for then a = and f A = f A and p A — ir A and the notion of 
consistency described in 1 3 1 holds because the relationships here are in effect the same 
as in (3). The following theorem shows that these quantities are approximately equal 
for any given finite temperature for values of /j > or a > sufficiently high. 

Theorem 1 Let f A and f A be defined as in \31\ and \32\ . respectively with a > 0. 
Then for values of fi£ A or a sufficiently high, 

fA 



fA 



l + af A 



Proof. 

This statement is proved by showing that for any e > 0, there is an energy value 
/* = mini/,} or an a > sufficiently high such that 

i£A 

T„ M Pih _^A <e (33) 

PA 1 + af A 
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For notational clarity, let LUi — : 
the limit as /* — » oo we obtain 



— and hence V 

Pa t-'l 



uJi = 1. Rewriting (I33i and taking 




1 + a/i 1 + a E jG a 




) 



/ 




+ a 



) 



1 



1 







a 



a 



since for all i 6 A, fi > /*. For a > sufficiently high, both terms have limits of 
zero. Consequently, for all e > 0, there exists an /* or a > such that J33l > holds. ■ 
Theorem^ shows that in the high energy regions of the energy landscape it makes 
no significant difference if energy levels are aggregated first to obtain the value of Ja 
and then transforming that value to obtain f a or we transform the values fi first to 
obtain the fi and then aggregate them to obtain — either way, they both yield values 
close to 1 1 a. 

4 Symmetry Relationships 

Tsallis observes a symmetry relationship in using the different constraints described 
earlier in Section |2] and in text surrounding l|6) 1 13 1. Tsallis' original incarnation of 
non-extensivity was based on his Type 1 constraint YliPifi = (see I13I - ) for 
stationary probability 



to the expected value of the eigenvalues of the Hamiltonian of the system although 
here, we can simply refer to it as the expected value of some energy). Tsallis points 
out that the form of pi is essentially the same as that of pi(t) except that 1 — q replaces 
every occurrence of q — 1 and vice versa including in the exponents (compare J34i with 
This interesting fact provides hints of additional symmetry relationships. In this 
section, these types of symmetry relationships are further explored in light of the scale 
invariance properties already developed. 

4.1 Scale Invariance Using Other Constraints 

It is quite reasonable to ask whether the constraint YliPifi = ^ <^ so leads to scale 
invariant forms and other symmetry relationships such as those described above. De- 
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fine the energy transformation 

^ 9 ' t)E i + (^r^ (35) 

where, for convenience, we use the earlier definition of a and will often denote this as 
simply /j. Using the same approach as in equations d!2i through dl6> we obtain 



dpi{t) _ pi(t) 



fi(t) (/)(*) 



dt t 2 

which again, using the same arguments on the aggregation of states, leads to 

dp A {t) _ pA{t) 



dt t 2 



f A (t)-(f)(t) 



Note that the only difference between this and the earlier result is that every occurrence 
of pi and fi is replaced with a pi and respectively. 

4.1.1 Second Moments 

Proceeding in the same fashion as in (120b through J25l > we obtain the result 

d(f) n = (/ 2 )o-(/)o + (l-g)(/ 2 )o 
dt t 2 

- + T 2 ■ 06) 

Scale invariance in second moments with the Tsallis constraint Type 1 is indicated by 

dh (f 2 )A-{f} 2 A + (l-U)(f 2 )A 



dt t 2 

■) . 

(37) 



t 2 i 2 

where d36i and (I37> are similar to d23l > and (125 > except that, as before, every occurrence 
of q — 1 and / has been replaced with a 1 — q and /, respectively. 

4.2 Probabilities and Energy Relations 

A number of additional forms of symmetry become evident when we examine the var- 
ious relationships between probabilities and transformed energy values. First, suppose 
we wish to define Pi(t) in terms of fi. Using the energy transformation function in dl5l 
it is easy to see that 

/■ = T7(3)I = I^ 

where again we use a = (^r^) for notational convenience. Note that the q — 1 in dl5> 
has been replaced with 1 — q in d38l >. 
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Note that (I35i has a similar form as J38i where fi is defined in terms of fi, and 
hence it follows that 

fi fi 



fi 



1 + (^r 1 ) /< 1 + aft 



and has the same form as the definition of fi in J15I . 

Now, substituting fl38l > into the form for pi(t) in 0, we get 



Pi(t) = 



1 + a 



1-a/i 



1 

1-q 



which, interestingly, is exactly the same form as in j34t above where the Type 1 con- 
straint was used. This also clearly implies that 



Z„ 



w 

E 



w 

E 



which further suggests another important relationship described below. Before this re- 
lationship is shown, however, a little more background into the use of other constraints 
is in order. 

The expression pj arise in a number of instances, in particular, in the definition of 
S q itself and so has a very fundamental character. It also arises in other fundamental 
relationships concerning conditional probabilities and Shannon Additivity (see |2|). It 
also became evident that it was useful to incorporate them in the constraints that define 
the stationary probability. The "first choice" using Tsallis' notation, YliPifi = ^ 
lead to problems. The "second choice", i.e., Y^iPifi = resolved some of them. 
Tsallis summarizes: 

The first choice was very little used in the literature because quite quickly 
it became obvious that it could not solve relevant mathematical difficul- 
ties existing in the approach of anomalous phenomena such as Levy su- 
perdiffusion. The second choice has been profusely used in the literature, 
and was not quickly abandoned because the deep physical reason for the 
generalization (g ^ 1) was not transparent. But, in the light of recent 
developments [ ] showing the relationship of the formalism with a pos- 
sible violation of the usual ergodic mixing hypothesis, features like the 
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g-expectation value of unity not being equal to one became clearly unac- 
ceptable. Then naturally emerged the third choice, which we believe to be 
fully satisfactory. . . 

1131 . We leave the implications of the third choice to future efforts. 

Tsallis notes the effects of the exponent q in the type 2 constraint as they "[priv- 
ilegiate] the rare and the frequent events" depending on whether q < 1 or q > 1, 
respectively 1 13 p. 535]. But this notion of shifting the probability weight of different 
energy values is, in some sense, equivalent to transforming the energy values them- 
selves as the following lemma illustrates. 

Lemma 1 For all q > 1, 

i i 

Proof: 

It follows from the definition of pi, that for all i, 

Now observe that j- 2 -^ = — — 1. Consequently for all i, 

Pi 

Zf 1 [1 + (¥) /<] ' 
Substituting this into ( I39l > and simplifying we get 

Pifi = Z Y q Pifi 
and by summing over all i, the result follows. ■ 

It is worth noting that this is consistent with the Legendre structure as noted in 1131 
p. 539] where the temperature t = \/{kj3). 

Finally, it is worthwhile to investigate the relationship between the constraint Type 
1 and the energy transformation function. Using a similar approach as before, we state 
the following lemma: 

Lemma 2 For all q < 1, 

w w 

z'^j2pifi = J2^ 

»=1 i=l 

Proof 

This follows using the same approach as in Lemma^ B 



15 



5 The Transformed Energy Landscape 



The appearance of the form for /j in dl5l raises a number of intriguing issues and pos- 
sibilities. In exploring these issues, it is helpful to gain some sense of what happens to 
the values of /,; relative to the values of /$ for different values of the entropic parameter 
q and temperature t. First note that for q = 1, fi = fi for all i and t > and produces 
the straight line depicted in Figure ^ For values of q > 1, values of fi are bounded 
above in that lim/^oo / = — ^j- and produces a monotonically increasing curve de- 
picted in Figure ^( see also Lemma |3J>. The ^-transformation reduces each value of 
energy in rough proportion to its magnitude. This has the effect of 'flattening out' the 
energy landscape when q > 1 as depicted in Figure|2]for several different temperatures 
with q = 2. It is worthwhile to note that for values of q > 1, the rank order of all 
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2 4 6 8 10 12 14 16 18 20 22 24 



Figure 1: Plot of / versus /. The transformation of the energy landscape is based on 
q = 1 (with ■ on straight line) and q — 2 (with ♦ below the straight line) for a fixed 
value of t = 10. 

values of / and / are preserved as the following lemma shows. 

Lemma 3 For all values of a > (i.e., q > 1 and t > 0) and for all positive reals x 
and y, 

x y 
x >y^— — > — — • 
1 + ax 1 + ay 

Proof 

For positive a, x and y, x > y if and only if x + axy > y + axy. Simplifying and 
dividing each side by 1 + ay and 1 + ax (on the right), the result follows. ■ 

It is also easy to see from Figure|2]and Lemma[3]that any curve at a given temperature 
majorizes the corresponding curve at a lower temperature. 
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Figure 2: The energy landscapes of / (with ♦ at top), and for q = 2, the values of / 
at t = 100 (with ■. 2nd curve), at t = 10 (with ▲, 3rd curve) and at t = 1 (with x at 
bottom). 



The net effect of the energy transformation as indicated in the preceding can be 
summarized in the following way: it smoothes out the landscape, i.e., reduces the relief 
of the landscape and lowers the energy values yet preserves the essential relational 
features of the landscape (in terms of rank order). 

5.1 Relating Exponentials and Powerlaws 

This energy landscape transformation provides several alternative perspectives on power- 
law distributions. The flattened landscapes in Figure|2]may explain the heavy tail dis- 
tributions associated with complex phenomena — the flatter energy landscape makes it 
easier, in some sense, to move to higher values of fi in the energy spectrum. For ex- 
ample, in using the classical Metropolis Algorithm and its associated exponential form 
using values of fi, uphill moves are more probable and this leads to a heavy-tailed 
"steady-state" distribution. 

To make this precise, the following theorem shows how this energy transforma- 
tion in effect parameterizes an exponential distribution and permits it to change into 
a power-law distribution without the necessity of taking limits. This is the oppo- 
site of what Tsallis describes as his "^-exponential" (see e.g., [13 p. 537]) where 
he states a power-law distribution that in the limit becomes exponential (recall that 
lim g ^i pi — TTi). Thus, in the limit a power-law takes on an exponential form. Here, 
we use the standard definition of an exponential form which asymptotically becomes a 
power-law. 
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Theorem 2 Let a > and x be such that ax 1 > 1 and define 



1 + az 7 



for some 7 > 0, then for any A > 

,-Aa 



-7 



e "~ — Ci ~ C2X 
as ax — > 00 where the constants C\ — e~ A / a and C2 = a~ 2 A e~ A M 



(40) 



Proq/: 

First note that lira^oo Ax = A/a, hence, for any fixed a > both sides of J40i have 
limits of as x — > 00. To show that e _A:r — e _A / a decreases to zero asymptotically as 
1 /a; 7 (within a constant), first note the identity for any real x and 7 > 0, 



1 



1 



■ x^ - x 37 + x^ 



l + x"f 

Dividing the numerator and denominator of x by ax 7 , we obtain 



1 



1 + air 7 



1 



1 + 



(41) 



(42) 



Using the general relationship in (1411 in the parenthesis in 1421 and substituting into 



yields 



,-X/a 



A 


1-J- 

ax 1 


1 


a 


+ (ax 7 ) 2 


A 


A 


A 


a 


1- —0 


a J a; z 


exp 


r A 


A 


[a 2 x 7 


a 3 x 2T 



(43) 



Note that since 7 > and ax 7 > 1, the series in d43i is absolutely convergent (using 
the ratio test), hence converges. Substituting a Taylor Series expansion of the exponen- 
tial in J43i (the second factor) yields 



-A/a 



1 + 



A 



A 



a 2 x 7 



A 2 



a 4 x 27 



(44) 



where we note that the higher order terms of the series in (1441 all have powers higher 
than a 4 £ 27 in the denominators (these are not shown). To show that er Xx falls off 
according to a power law, we evaluate the expression 



lim 

ax — >oc 



-AM 



a- 2 Ae- A M- 7 



(45) 



to assess its asymptotic behavior and the rate at which it approaches its limiting value 
(if any). Substituting J44l> for the numerator of d45l > and multiplying the numerator by 
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x 1 (from the a; 7 in the denominator) and reordering the terms of the Taylor Series 
expansion to indicate only those with powers of a 4 x 27 or less yields 



x 7 e- A/a 



A 



A 



A 2 



a 2 a; 7 
Ae" A / a 



a 3 x 27 
1 



2a 4 x 27 
A 



x 1 e- x/a 



in the numerator. Consequently, 



,-Aa 



ax^ 2a 2 x~t 



,-X/a\ 



lim ■ 

ax — »oo 



a- 2 Xe~ x / a 
1 

ax 7 



lim 

ax — >oc 



1 



A 



2a 2 x"i 
lim [1 - E(a,x)] = 1 



ax — >oc 



as ax — > oo and hence J40i 



where the discrepancy E(a.x) = ^ - 
follows. ■ 

Figure[5]compares the decays of e~ Aa: — e~ x / a and A e~ x / a / (a 2 x 7 ) for the given 
values of a and A and shows that the curves are virtually coincident even for the rela- 
tively low values of x indicated in the figure. 
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5.2 A Recursive Transformation 

The form of the ^-transformation of energy also permits stating an interesting recursive 
feature that may be useful in relating the level of aggregation to the entropic parameter. 
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Recall that the scale invariant properties indicated in H9i and J25t are based on aggre- 
gating energy levels. From these aggregations, several scale-invariant relationships can 
be defined involving the transformed energy values fa defined in J17I . But this value is 
associated with an aggregation of particles, each with its own value of "energy" where 
this aggregation may actually result from a series of successive aggregations because 
the g-transformation has this recursive aspect. This recursive feature of the energy 
transformation in (1151 is based on the following relationship where we substitute fi for 
fi in J17I . This results in 

#*.x = (46) 



1 + afi 



For consistency in notation, let f7 Q = f? and it follows that fci = fi for 7 = 1 as 
defined earlier (see J15IT ), Using these relationships, the following lemma holds. 

Lemma 4 The k"' energy transformation raised to any power 7 > is related to 
fi as follows: 



fi 

%k = 1 + kaf? (4?) 



Proof 

This is an obvious implication of J46l > and the use of the induction method. ■ 

This suggests there is some equivalence between the number of aggregations and 
the value of the entropic parameter q. That is, the coefficient of fi, ka, in the denom- 
inator of 1471 corresponds to a higher value of the entropic parameter. Successive (or 
larger?) aggregations of energy are, in some sense, equivalent to larger values of q. 
These asymptotic results and the recursion described in Lemma 0] may provide new 
perspectives on power laws and the nature of critical phenomena which are discussed 
below. 



6 Conclusion and Future Research 

This article has explored how various statistical components of a non-extensive system 
exhibit forms of scale invariance. By aggregating energy levels and defining certain 
statistical quantities for these aggregated sets in appropriate ways, such as its station- 
ary probability and expected energy levels, various operations on these quantities result 
in identical mathematical forms as for the corresponding quantities associated with in- 
dividual energy states provided they are based on a transformed energy value. This 
energy transformation appears in a number of scale invariant and symmetry relation- 
ships. 

The manifestation of the energy transformation in these scale invariant forms is a 
surprising element and bears further investigation. The manner in which it appears sug- 
gests that scale invariance exists only by virtue of an energy transformation function. 
This suggests a view that information (entropy) loss may occur between aggregated 
states where there has already been some information loss due to prior aggregations. 
Thus, if one can make the leap from aggregating a system of components to aggre- 
gating their associated energy levels, it may provide insight into a number of critical 
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phenomena relating to power-law behaviors. Thus, aggregated system components 
may further combine to form yet larger aggregations where there is a further loss of 
entropy due to non-extensivity. But from the recursive property in0] there is in some 
sense an equivalence between a succession of aggregations and changes in temperature 
and/or the entropic parameter q. Thus, the level of aggregations in particular parts of a 
large systems can be seen as equivalent to differences in the temperature, heat capacity 
and so forth between these different parts of a large system in a far-from-equilibrium 
regime. 

This may also provide some insight into why certain types of systems have particu- 
lar values of critical exponents. For example, in the border between order and disorder, 
information loss may occur in connection with how inelastic collisions manifest them- 
selves during a phase transition. But successive aggregations may happen for only a 
certain fraction of the energy states in a system or the tendency towards aggregation 
may change with the size of aggregated states. Successive aggregations and the atten- 
dant loss of entropy (when q > 1) may then lead to a certain distribution in terms of 
the size of these aggregated states indicated by a power-law. The relative frequency 
of the size of aggregated states and their dependance on macroscopic properties could 
then explain the particular values of critical exponents as reflecting the average number 
of aggregations in large systems. In this sense, the universality of these critical expo- 
nents suggests that there may be a typical number of successive aggregations with a 
distribution of this number that reflects and explains the non-integer values of critical 
exponents. 

A number of these issues are further explored and developed in 1 9 1 where a gen- 
eralization of Tsallis' entropy definition sheds some additional light on these energy 
transformations. Transformation families are explored as well as recursive relation- 
ships similar to those described above. In addition, we use the generalization of Tsallis' 
entropy to provide more general ways to state power law distributions in exponential 
forms, a reverse approach to Tsallis where he provides a mechanism where a power 
law form can be made asymptotically equal to an exponential form. This also provides 
clues as to how to take fuller advantage of these results in modeling complex systems 
and developing simulation models. New approaches for the simulation of complex 
systems and ways in which to overcome the so-called "broken ergodicity problem" are 
also explored (see 1 8 1). 
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